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ABSTRACT 


Semi-automatic segmentation is common in medical image processing 
because anatomical geometries demonstrated by human anatomical parts 
often requires manual supervision to provide desirable results. However, 
semi-automatic segmentation has been infamous for requiring excessive 
human intervention and time consuming. In order to reduce aforementioned 
problems, seed labels have been generated automatically using superpixels in 
our previous works. A fixed threshold method has been implemented to 
classify cartilage and background superpixels but this method is reported to 
lack the adaptiveness to changing image properties in 3D magnetic resonance 
image of knee. As a result, the coverage of background seeds are not 
sufficient to cover whole background area in some cases. In this work, 
we proposed a local mean based adaptive threshold method as a better 
alternative to the fixed threshold method. We calculated local mean for each 
block in an integral image and then use it to differentiate background 
superpixels from cartilage superpixels. The method is robust to illumination 
changes and simple to use. We tested the adaptive threshold on 35 knee 
images of different anatomical geometries and proved the proposed method 
could provide more comprehensive background seed labels distribution 
compared to fixed threshold method. 
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1, INTRODUCTION 

Thresholding is a common image processing technique used to classify unlabeled pixels into 
different classes [1]. In general, there are two types of threshold approach in image processing 1.e. global 
threshold and local threshold. Global threshold [2], also known as fixed threshold, is the most simple and 
straightforward threshold technique used for binary image segmentation by applying a general threshold 
value across the image. An effective fixed threshold method will require discriminative foreground and 
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background image properties. Meanwhile, local threshold, or adaptive threshold, will implement several 
threshold values to different regions/ classes of the image. The threshold values will be determined by the 
region information of the neighboring pixels where the thresholding 1s applied. 

One popular local threshold is Wellner’s Method [3]. The local threshold method 1s straightforward 
and robust to variation in illumination, which is an important characteristic of medical images. An average of 
a predefined window is computed and defined around each pixel. Neighboring pixels of all sides are being 
considered to obtain a better average of image pixels. Then, integral image is applied to average the 
computation of image pixel in linear time. Compared to fixed threshold, adaptive threshold 1s more adaptive 
application-wise due to its flexible threshold setting. Therefore, the application of threshold method is wide; 
ranging from medical image segmentation [4], computer-aided diagnosis [5] to image denoising [6]. 

The segmentation of knee cartilage is challenging [7]. While automatic segmentation is desirable, 
it is not feasible due to the irregular anatomical geometry and thin structure of knee cartilage. Huge amount 
of training data will be the prerequisite to produce successful automatic segmentation models, in addition to 
heavy computational burden. As a result, semi-automatic knee cartilage segmentation methods have been 
viewed as the better alternative to automatic segmentation attributed to its direct expert intervention 
advantage to attain desirable results. There are several semi-automatic segmentation methods such as graph 
cuts [8], livewire [9] and random walks [10] which are applied in knee cartilage segmentation. 

Nevertheless, most semi-automatic segmentation methods have reported major technical limitations. 
Graph cuts method has been sensitive to image noise and is usually limited to binary segmentation, which is 
not suitable for multilabel knee cartilage segmentation [11]. Livewire method is originated from Shortest 
Path algorithm but the algorithm is plagued by the “shortcut” problem. In order to overcome this limitation, 
excessive boundary points are required to guide the segmentation. This will be extremely tedious especially 
when handling the curvature region of cartilage. Finally, random walks method is robust to image noise and 
shortcut problem, in addition simultaneous multilabel segmentation. Unfortunately, the method is infamous 
to be sensitive to location of seed labels. 

In our previous works, an automatic seed label generation model has been developed to overcome 
the problem faced by random walks segmentation. Fixed threshold methods has been utilized in the model to 
differentiate background from cartilage seed labels. Yet, fixed threshold method is not robust to the varying 
image superpixels’ intensity properties. Hence, the objective of this study is to improve the automatic seed 
labels generation model’s thresholding mechanism with Wellner based adaptive threshold. 


2. MATERIALS AND METHOD 

Knee image structure is complex since it consists of background and foreground objects which is 
tibial, patella and femoral cartilage as a whole. Non-cartilage background usually occupies most of the image 
and the knee cartilage is always positioned at the center of image [12]. Development of automatic seeds 
generation aims to substitute the background scribbling required to initialize conventional random walks 
segmentation [10-14]. For instance, random walks are extremely sensitive to placement of seed locations due 
to a lack of global feature distribution. Hence, we proposed to substitute manual background labelling with 
an automatic seed labels generation [15]. Figure 1 explains the advantage of the automatic system to 
overcome excessive user intervention. 





(b) 


Figure 1. (a) Scribblings are required to initialize the random walks segmentation, but it 1s time consuming 
and remains sensitive to seed locations. (b) Use of background seeds saves scribbling time and 
reduce user interventions 
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2.1. Limitation in Automatic Seeds Generation 

While the automatic model [16] helps to reduce the excessive manual intervention problem, it is not 
without technical limitation. Currently, the model’s fixed thresholding is derived from averaged histogram of 
superpixels’ saliency values. The fixed threshold value was set at T=100. But saliency detection often detect 
the object (cartilage) at the center of image; rendering the superpixels near to center location to possess 
higher saliency values. As a result, fixed threshold method is not robust enough in some occasional cases and 
create an empty “hole” phenomena in the center of image. Figure 2 demonstrates abovementioned 
phenomena in which the model fails to recognize background superpixels near to the center of image. 





(b) (d) 


Figure 2. (a) - (d) Occasional failure cases encountered when using the automatic seeds generation model 


2.2. Local Mean-Based Adaptivr Threshold 

In this works, we replaced the fixed threshold with a local mean-based adaptive threshold using 
integral image. Instead of pixelwise intensity value, we used saliency value for each superpixel as threshold 
parameter [17]. Figure 3 demonstrates the feature map of superpixel-based MR image of knee using saliency 
values. An integral image, I is also known as a summed-area table where the cumulative sum of real numbers 
1s computed over a rectangular region of the image. 





(b) 





Figure 3. (a) First round of saliency detection using cartilage prior. (b) Second round of saliency detection 
using background priors 


The saliency value, denoted as f(x,y) at any location within the integral image, /(x,y), is equivalent 
to the sum of all superpixels’ values above and to the left of the superpixel, including the corresponding 
superpixel. The cumulative summation of all (x,) terms at each location (x,) is given in (1). 


I(x,y) = f(x,y) + I(x — ly) + Iy — 1)- Tx — Ly — 1) (1) 


The main idea of this adaptive threshold is to compare each superpixel to an average of the 
surrounding superpixels. Let’s set P(x,y) as the original superpixel saliency value at superpixel (x,y) within 
an image of size mxn. Then, a window with pre-defined two dimensional rectangular area w is implemented. 
In this work, we defined the size of window, w=15x15 and computed the mean value of each saliency value 
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based on the integral image. The saliency values were averaged by the window. We set the superpixel 
threshold, T(x,y) on whether it was lower than the t percent of average value of pixels within w. In case of 
saliency value lower than the threshold, it was classified as 1 (black/ background) and vice versa. 
The threshold rule is given in (2). 


: f(%y) 100-t 
T(x,y) = iy Oy) < ( w )( 100 (2) 
0; otherwise 


Figure 4 shows the use of integral image to compute the sum of pixels’ values and the result of 
adaptive threshold. 





(a) (b) 


Figure 4. (a) Example of integral image produced from MR image of knee. (b) Example of binary image 
classification as a result of adaptive threshold 


2.3. Automatic Generation of Seeds 
After the classification of superpixels based on adaptive threshold, we ought to generate seeds on 
each background superpixel. To do so, Fuzzy C-Means (FCM) [18] was implemented to generate seeds. 


The objective function of FCM, J,,,was governed by a membership Mi in (3). 


Jm = Mier U5 a1 WT? Ilxi — Cjll? (3) 
Where the membership uj; evaluates the percentage of belonging to pixel x; to cluster C; using Euclidean 


distance, dj; = IY xX; - c and m is the degree of membership. 


Even though the cluster centroid did not directly represent any seed, we could approximate the 
nearest pixel to the centroid as seed using Euclidean distance as shown in (4). 


dist (ij) = min IY x, — C;° (4) 


A demonstration of seeds generation using the adaptive threshold classification is shown in 
Figure 5. We still reserved cartilage label for manual scribbling as human knowledge priori is critical to 
produce desirable results. The implementation of this work was conducted by using an inhouse developed 
medical image processing software [17-19]. 


3. RESULTS 
3.1. Image Dataset 

The knee images are taken from the Osteoarhtritis Initiative (OAI) dataset. Consents were obtained 
from all patients and approved by the Institutional Ethical Committee. A total of 35 knee images were used in 
this study. All images were acquired using 3.0T MRI scanner (Siemens Magnetom Trio, Erlangen, Germany) 
with quadrate transmit-receive knee coil (USA Instruments, Aurora, OH) [22]. The DESS images have 
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section thickness of 0.7mm and in-plane resolution 0.365 x 0.456 mm” (field of view = 140x140 mm, 
flip angle = 25° ,TR/TE = 16.3/4.7 msec, matrix size = 384x384 mm, bandwidth = 185 Hz/pixel) [23]. 


The segmentation model was developed by using MATLAB R2014a (Mathworks, Natick, MA) on a laptop 
equipped with Core 17-4700HQ@2.50 GHz processor and 8.00 GB RAM. 





(b) 


Figure 5. (a) Example of seeded image using adaptive threshold method, (b) Segmentation result of femoral, 
tibial and patellar cartilage 


3.2. Qualitative Results 

The effect of the proposed method is evaluated by comparing the distribution of background seed 
labels generated by using adaptive threshold method against fixed threshold method across the knee image. 
The aim of the qualitative evaluation is to examine the capability of the proposed adaptive threshold method 
to overcome the empty “hole” phenomena and fully enclose the background region with computer generated 
background seed labels; thus minimizing the need of manual intervention. The results of 10 knee images 
indicating before and after the implementation of the proposed method are demonstrated in Figure 6. Since 
the anatomical geometry of knee cartilage varies across image slices, we have included the implementation of 
adaptive threshold method across different knee image slices. 


4. DISCUSSION 

Based on the results, we can observe obvious improvement in background seeds label distribution 
across the knee image when adaptive threshold method was applied to the automatic seed labels generation 
model. The reliability of the implementation is confirmed by applying the method across different knee 
cartilage geometries. Comparatively, automatic seed generation model using fixed threshold method often 
fails to create background labels at the center location of the knee image. This failure is mainly due to the 
higher saliency values demonstrated by background superpixels located near to the center position. 
Therefore, fixed threshold method fails to recognize those background superpixles with higher 
saliency values. 

On the other hand, automatic seeds generation model using adaptive threshold method can easily 
resolve the phenomena reported by the fixed threshold method. As shown in Figure 6, background 
superpixels located at the center of image have been covered with background seeds successfully. 
For instance, adaptive threshold method has been found to be more robust to varying image property such as 
shape, intensity, the presence of noise and uneven background. The fixed threshold is more effective when 
the intensity distribution between knee cartilage and background are clearly distinct and thus, the selection of 
a general threshold value can fulfill the overall image property condition. In this context, we can observe the 
use of Wellner based adaptive threshold proves to be more adaptive to varying saliency values possessed by 
the background superpixels located near to the center of image, where the cartilage superpixels are located. 

Although our study has suggested that the adaptive threshold method can provide better seeds label 
distribution across the image background, there are several limitations in this study. First, there is no 
categorization of knee image into healthy and abnormal knee cartilage. This is attributed to the focus of this 
paper is to assess the distribution of background seeds instead of cartilage. Hence, the influence of 
pathological features such as osteophytes, bone marrow lesions and effusion are minimal. Second, we did not 
conduct quantitative analysis in this paper. Nonetheless, the absence of quantitative analysis in this study still 
follows strictly the standard evaluation protocol of threshold-based analysis from existing studies [24-25]. 
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difficult to apply evaluation parameters such as 


Besides, the aim of this study is determine the capability of the adaptive threshold to provide comprehensive 
Coefficient (DSC) which requires at least two object classes. Nonetheless, qualitative analysis by comparing 


the implementation of adaptive threshold and fixed threshold method has demonstrated the superior 


performance reported by the adaptive threshold method. 
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Figure 6. Effect of adaptive threshold on the automatic seeds generation model. (a), (d), (g), (J), (m), (p), (Ss) 
and (v): Original MR image of knee. (b), (e), (h), (k), (n), (q), (t) and (w): Seeds distribution using fixed 
threshold method. (c), (f), (1), (1), (0), (r), (u) and (x): Seeds distribution using the adaptive threshold method. 


5. CONCLUSION 

The study concluded the successful use of adaptive threshold as a better alternative to fixed 
threshold method as implemented previously in out automatic seeds generation model. The advantage 
illustrated by the adaptive threshold method minimizes the need for additional manual intervention. 
Future works should focus on conducting further quantitative analysis to strengthen the advantage of adaptive 
threshold and also expanding the automation of seed generation to different types of knee cartilage 1.e. 
femoral, tibial and patellar cartilage. 
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